Schedules as automations - #2293
Conversation
The scheduling job creators accept an optional trigger dict (stored as job meta data), like the forecasting pipeline already does. The API trigger endpoint records origin API; the CLI and automations follow in the next commit. The status page's 'Created Via' column picks this up automatically. Part of #2288 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Rbix8k1JfeUWNXEmHEZVpX
Automations now also support the 'schedules' type: - `flexmeasures add automation --type schedules` validates the parameters as a schedule trigger message (per the AssetTriggerSchema, as accepted by the API trigger endpoint, without the asset id). The schedule 'start' may be omitted, in which case each run schedules from the run time (floored to the message's resolution, if given) — a fixed start draws a warning. - The runner dispatches schedules automations to the same job creators as the API trigger endpoint (sequential or simultaneous), recording trigger meta data (origin automation) on the queued jobs; `flexmeasures add schedule --as-job` now records origin CLI. - Job stats for schedules automations are counted from the scheduling job cache (asset-level wrap-up jobs and per-sensor device jobs). - The UI automations page's Schedules tab is now enabled, with automations filtered by type per tab. Part of #2288 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Rbix8k1JfeUWNXEmHEZVpX
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Rbix8k1JfeUWNXEmHEZVpX
Documentation build overview
135 files changed ·
|
Context: - PR #2293 is stacked on the reviewed forecast automation branch. Change: - Merge the reviewed parent and preserve schedule automation behavior across the five semantic conflicts. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Context: - Schedule automations do not use a data generator, but the reviewed forecast automation schema required one. Change: - Make the foreign key nullable while retaining a database check that forecasts always have a generator. - Add a forward migration without weakening data-source deletion semantics. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Context: - Review uncovered untested forecast-only options, invalid durations, and DST start calculation. Change: - Add CLI and trigger-preparation regressions while retaining forecast sensor validation. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Context: - Persistence, inherited flex configuration, descendant statistics, and job counts lacked realistic coverage. Change: - Move automation service tests under the fresh-database fixture and add end-to-end schedule regressions. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Context: - API provenance was not asserted across asset and sensor schedule jobs. Change: - Verify API trigger metadata on sequential descendants, wrap-up jobs, and sensor jobs. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Context: - The asset automation page tests still targeted the former single table. Change: - Assert type-specific tables, error rendering, filters, and hidden-tab column adjustment. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Context: - Minimal asset schedules lost stored defaults, invalid timing could execute, provenance was incomplete, and descendant jobs were miscounted. Change: - Validate and floor fixed durations safely, inherit stored flex configuration, preserve provenance, and count the jobs actually dispatched. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Context: - Schedule automation creation silently accepted forecaster settings that could never affect scheduling. Change: - Detect supplied forecast-only options and return a user-facing usage error. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Context: - DataTables initialized in the hidden schedule tab could render with stale column widths. Change: - Add tab accessibility state and adjust initialized table columns when a tab becomes visible. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Context: - The feature guide linked only to the API root and omitted fixed-start and duration constraints. Change: - Document canonical fields, runtime start behavior, timing validation, and generator-free schedule automations. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Context: - Manual testing exposed raw parser exceptions for malformed automation files. Change: - Require a user-facing usage error for invalid YAML in both config and parameter files. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Context: - PyYAML parser errors escaped automation creation without a useful CLI message. Change: - Translate malformed config and parameter files into a normal Click usage error. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Context: - Manual execution showed minimal automations failing for a one-device asset tree. Change: - Exercise both simultaneous and sequential dispatch using realistic flex configuration stored on the child asset. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Context: - A one-device asset tree collapsed to sensor scheduling without a sensor, and sequential dispatch could not resolve its stored output. Change: - Preserve asset-triggered flex models as a list and resolve sequential device sensors from stored consumption or production outputs. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Context: - Scheduling service docstrings did not distinguish single-job and sequential provenance behavior. Change: - Document where trigger metadata is stored and correct the simultaneous return description. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Brings in the timezone and catch-up work from #2396, the sensor links, the sortable listing and the review fixes from #2290. Resolved by keeping both sides, so an automation carries a timezone and a scheduling cursor as well as its input and output sensors, and the job statistics still scan the scheduling queue for schedule automations while the forecast path is unchanged. Two resolutions went further than picking a side. make_cli_options_optional is gone: add_cli_options_from_schema now takes force_optional, which relaxes the same requirement and also hides the forecast options, which is what a schedule automation wants anyway, since those options only apply to forecasts. The asset page test no longer expects a single automations table to be hidden, as the listing is now one table per automation type. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Merging the timezone and catch-up work alongside the schedule automations left two alembic heads, one adding an automation's timezone and scheduling cursor and one allowing a schedule automation without a data generator, so flexmeasures db upgrade refused to run and the Docker image build failed. The two touch different columns, so the merge point has nothing of its own to do. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com> # Conflicts: # flexmeasures/data/services/automations.py
Prepare the scheduling configuration through the same scheduler collection path used before queueing, then derive declared input and output sensors so generator-free schedule automations expose their stored flex dependencies and participate in sensor relationships. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Exercise a minimal schedule that inherits its flex model and context from the asset tree, confirming that price inputs, schedule outputs, and the sensor-to-automation relationship are all reported from stored configuration. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Create a generator-free schedule with stored price and output sensors and assert that the details endpoint returns both dependency sets instead of reporting empty arrays. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
…differ from their default The guard against combining forecaster options with --type schedules compared the forecaster against its default, so naming the default forecaster explicitly passed silently and the automation was created, leaving the impression that the option had applied to a schedule automation. Ask which options were actually given on the command line instead, the way the --source conflict check already does, and name the offending options in the error rather than listing every option it could have been. Signed-off-by: Mohamed Belhsan Hmida mohamedbelhsanhmida@gmail.com
Manual test walkthroughEverything below was run against a development database on this branch. It assumes the forecast Pick an asset that describes its own flexibility, so a minimal trigger has something to work with — flexmeasures jobs run-worker --name schedules-demo --queue scheduling1. A minimal trigger is enoughThe parameters of a schedule automation are a trigger message: what echo 'duration: "PT12H"' > trigger-message.yml
flexmeasures add automation \
--asset 25 \
--name "Hourly schedules" \
--cron "0 * * * *" \
--timezone Europe/Amsterdam \
--type schedules \
--inactive \
--parameters trigger-message.yml
# → Successfully created inactive automation 'Hourly schedules' (ID: N) to compute schedules
# for asset 25, recurring per cron string '0 * * * *' in timezone 'Europe/Amsterdam'.Note it needs no forecaster and no data source: 2. The schedule starts when the run startsOmitting A fixed echo 'start: "2025-11-16T00:00:00+01:00"' > trigger-fixed.yml
echo 'duration: "PT12H"' >> trigger-fixed.yml
flexmeasures add automation --asset 25 --name "Fixed window" --type schedules \
--inactive --parameters trigger-fixed.yml
# → Warning: the schedule 'start' is fixed, so each run will compute the same period.
# Omit 'start' to schedule from the run time instead.
# → Successfully created inactive automation 'Fixed window' (ID: M) ...3. Forecaster options do not applyA schedule automation is not computed by a forecaster, so anything configuring one is refused and flexmeasures add automation --asset 25 --name "Wrong options" --type schedules \
--parameters trigger-message.yml --forecaster SomeOtherForecaster
# → Error: --forecaster cannot be combined with --type schedules: a schedule automation is not
# computed by a forecaster.
flexmeasures add automation --asset 25 --name "Wrong options" --type schedules \
--parameters trigger-message.yml --train-period P7D
# → Error: --train-period cannot be combined with --type schedules: ...Naming the default forecaster is refused as well, since it was still given: flexmeasures add automation --asset 25 --name "Wrong options" --type schedules \
--parameters trigger-message.yml --forecaster TrainPredictPipeline
# → Error: --forecaster cannot be combined with --type schedules: ...The check asks which options were passed rather than comparing their values against their defaults, A parameters file that is not a mapping is reported plainly rather than failing later: echo 'just a string, not a mapping' > bad.yml
flexmeasures add automation --asset 25 --name "Bad yaml" --type schedules --parameters bad.yml
# → Error: The --parameters file must contain a YAML or JSON object at the top level.4. Run it, and see a scheduling jobflexmeasures edit automation --id N --cron "* * * * *" --activate
flexmeasures jobs run-automations
# → Automation N ('Hourly schedules') queued 1 scheduling job(s) for asset 25.The job lands on the scheduling queue, not the forecasting one, and carries the same provenance For a sequential schedule the automation queues one job per device plus the wrap-up job, and the job 5. The Schedules tabThe Schedules tab is now populated, alongside Forecasts. Each type has its own listing, and the 6. Clean upflexmeasures edit automation --id N --deactivate
flexmeasures delete automation --id N --force |
Description
Automations can now compute schedules on a recurring basis, alongside forecasts. The automation
still only decides when work is due and queues it; the existing scheduling machinery computes the
schedule and records it.
Parameters are a trigger message. A schedule automation stores what the
[POST] /assets/(id)/schedules/triggerendpoint accepts, without the asset id, validated againstAssetTriggerSchemawhen the automation is created. Use the canonical API field names, includingflex-model,flex-contextandforce-new-job-creation.The schedule moves with the clock. Omit
startand each run schedules from its own run time,floored to the message's
resolutionwhen given and otherwise to the minute. A fixedstartisaccepted, but then every run computes the same period, so the CLI warns when creating such an
automation.
Minimal triggers inherit stored flex config. The flex-model and flex-context can live on the
asset tree instead of in the trigger message, so
duration: "PT12H"on its own is a completeautomation for an asset that already describes its own flexibility. The scheduler resolves the rest
the same way it does for a triggered schedule.
No data generator. A forecast automation points at a data source holding its forecaster
configuration; a schedule automation has nothing equivalent, so
generator_idis now nullable and adatabase constraint requires it only for forecasts.
Provenance. Scheduling jobs record how they came about in the same way forecasting jobs do:
{"origin": "automation", "automation_id": <id>}, so a scheduled job queued by an automation isdistinguishable from one triggered through the API or the CLI. Job statistics count scheduling jobs
for schedule automations, including the per-device jobs of a sequential schedule.
CLI.
flexmeasures add automation --type schedulestakes the trigger message through--parameters. Forecaster options are refused for schedule automations — the check asks whichoptions were actually given on the command line, rather than comparing their values against their
defaults, so naming the default forecaster is refused too instead of passing silently. A
--parametersor--configfile that is not a mapping is reported as such rather than failingobscurely later.
UI. The Schedules tab on an asset's Automations page is now populated, with one listing per
automation type. Reports remain a disabled placeholder until #2297.
documentation/changelog.rstLook & Feel
The Schedules tab is now populated, with its own listing alongside the forecasts one. Here an hourly
schedule automation on a battery, recurring in Europe/Amsterdam. Reports remains a placeholder until
#2297:
How to test
See the manual test walkthrough in the PR comments.
Further improvements
and then looking for the fields that name where results go. That agrees with the scheduler's own
resolution for the shapes that matter — a device's
sensor, and itsconsumptionandproductionoutputs — and errs towards reporting more rather than fewer, which is the safedirection once the same resolution guards who may create an automation (CRUD for automations in the API and UI #2294). Checking outputs
against what a scheduler actually returns at run time is tracked in Check the sensors a scheduler actually writes to, instead of predicting them when an automation is created #2421.
occurrence is worth scheduling at all once its window has passed — is still open.
Related items
Closes #2370. Part of the automations story #2334. Stacked on #2290, followed by #2294.
Sign-off